#model merging

'mmBERT is an encoder pretrained on 3 trillion tokens across 1,833 languages that runs 2–4× faster than previous multilingual encoders and supports 8k token contexts. It combines annealed language learning, inverse masking, and model merging to boost performance on high-resource and low-resource benchmarks.'

DeepSeek-TNG introduces R1T2 Chimera, a new Assembly-of-Experts LLM that delivers twice the speed of R1-0528 and improved reasoning, available now under MIT license.